2026 评测基准地图The 2026 Benchmark Map
微型指南 · 约 9 分钟Guide · about 9 min · 引用核验 2026-11-03（仅引用链接与日期） · refs checked 2026-11-03 (reference links and dates only)
榜单不是真理，但不同基准测的是不同能力。这张地图按编码、Agent、长文、多模态、安全分组，并给出读榜检查项。Leaderboards are not truth, but different benchmarks measure different abilities. This map groups coding, agents, long context, multimodal and safety, with a reading checklist.
编码：HumanEval 已偏老，SWE-bench 系列更接近真实仓库任务；看 Verified 子集与版本。
Agent：τ-bench、AgentBench、Terminal-Bench 测多步任务与工具调用；看任务集、工具面与运行次数。
长文：RULER、LongBench 测长上下文检索与推理；看长度档位与「中段」表现。
多模态：MMMU 测大学级图文推理，Video-MME 测视频理解；看模态覆盖与评测协议。
安全与对齐：看拒答、越狱、偏见与红队基准，但不要用单一分数替代你的业务验收。
读榜六问：版本、训练截止日、任务集是否公开、是否去污染、pass@k 还是单次、成本与步数是否披露。
结论：基准帮你缩小候选，不能替你决定；最终用你自己的任务集回归。
Coding: HumanEval is dated; the SWE-bench family is closer to real repository work. Check the Verified subset and version.
Agents: τ-bench, AgentBench and Terminal-Bench measure multi-step tasks and tool use. Check the task set, tool surface and number of runs.
Long context: RULER and LongBench measure retrieval and reasoning over long inputs. Check length tiers and middle-of-context behaviour.
Multimodal: MMMU measures university-level image-text reasoning and Video-MME measures video understanding. Check modality coverage and protocol.
Safety and alignment: refusal, jailbreak, bias and red-team benchmarks exist, but a single score cannot replace your own acceptance test.
Six questions for any leaderboard: version, training cutoff, public task set, decontamination, pass@k versus single run, and whether cost/steps are disclosed.
Conclusion: benchmarks narrow the candidates; they do not decide for you. Run your own task set before shipping.
例子与判断Example and interpretation
以下是教学情境，不是本站运行或实测记录。A teaching scenario, not a measured or executed site test.
准备做文档问答时，先找覆盖检索和有依据回答的任务，而不是只看代码生成分数。记录所用基准版本和日期，再用自己的文档补测缺口。地图用于找证据，不负责替你决定选谁。For document Q&A, seek retrieval and grounded-answer tasks rather than code-generation scores. Record benchmark version and date, then assess gaps on your documents. The map helps locate evidence, not select a winner for you.
基准测试Benchmark 模型评测的局限Evaluation Limits 数据污染Data Contamination
- SWE-bench（arXiv 2310.06770）SWE-bench: Can Language Models Resolve Real-World GitHub Issues? ↗
- τ-bench（arXiv 2406.12045）τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains ↗
- RULER（arXiv 2404.06654）RULER: What's the Real Context Size of Your Long-Context Language Models? ↗
- MMMU（arXiv 2311.16502）MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark ↗
AI 常见误区总索引AI Misconceptions: Index · 怎么判断 AI 新闻是不是炒作How to Tell if AI News Is Hype · 怎么评测 RAG 的效果How to Evaluate a RAG System · 怎么自建评测口径How to Build Your Own Eval